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APPARATUS AND METHOD FOR ROTATING factured with the Bayer pixel pattern format will be coupled 

THE DISPLAY ORIENTATION OF A to image processing routines specifically designed to accept 

CAPTURED IMAGE ' pixel signals in alternating sequences of "GRGRGR" and 

"BGBGBG." Due lo possible imperfections in the outer 

This application relates to co-pending U.S. patent appli- 5 rows and columns of pixels that make up the image sensor, 

cation Ser. No. 08/355,031, entitled A System and Method conventional digital cameras sometimes have image sensors 

For Generating a Contrast Overlay as a Focus Assist for An '^rge enough so that one or more lines of pixels at the sides 

Imaging Device, filed on Dec. 13, 1994, by inventor Eric C. of the image sensor can be ignored. 

Anderson; and U.S. patent application Ser. No. 08/384,012, Referring now to FIG. ID, a prior art graphical represen- 

entitled Apparatus and Method for Camera Image and 10 tation is shown of the object as captured in the portrait 

Orientation Capture, filed on Feb. 6, 1995, by inventor Scott orientation and output upon an image display 100. The 

FuUam. The subject matter of the two applications described image display 100 is typically a conventional stand-alone, 

above is hereby incorporated by reference. These related personal computer CRT having a top 102, a right side 104, 

applications are commonly assigned to Apple Computer, a left side 106 and a bottom 108. 

Inc. 15 As previously described, the image processing routines 

within digital cameras are conventionally designed to pro- 

BACKGROUND OF THE INVENTION cess pixel signals on a line-by-line basis according to only 

1. Field of the Invention one pixel pattern format. Thus, conventional digital cameras 
The present invention relates generally lo an apparatus P^°^^^^ ^"^^S^^ ^ ^^^y ^^^^ ^^^^^^ ^° ^ landscape format. 

and method for orienting an image. More particularly, the ^" presentation of the landscape image of FIG. lAupon 
present invention is an apparatus and method for rotating a ^^^^^ ^^^P^^^' "^0?" (i.e. "top portion") of the 
captured image to an orientation corresponding to an imag- landscape image would correspond to the top 102 of the 
ing subsystem's orientation at the time in which the image '"^^^^ display 100. Such an orientation for land- 
was captured scape images on the image display 100 is quite natural and 

2. Description of the Background Art acceptable for ease of viewing the image. However, the 

, . . , . o . presentation of the portrait image of FIG. IB Upon the Ullage 

When a digital camera captures an image of an object the ^-^^^^^ ^^^^ .jq^, ^^^^ p^^j^^jj -^^^^ 

camera s frame of reference with respect to the object spending to either the right side 104 or the left side 106 of 

produces a desired image orientation. Two conventional ^^^^ ■ ,3 depending on how the user had 

image onenlations exist, namely, a landscape orientation and 30 digital camera. FIG. ID explicitly shows the case 

a portrait orientation. Referrmg now to FIG. lA, a pnor art ^^ere the "TOP" of the portrait image corresponds to the 

graphical representation of an object in a landscape onen- ^ ^^^ ^ ^ j^q j^^^ "sideways" 

tation is shown in which the image's width is greater than orientation for portrait images on the image display 100 is 

Its height. Refernng also now to FIG. IB. a pnor art unacceptable and unnatural. TTiis undesirable by-product of 

graphical representation of the object in a portrait orientation 35 conventional digital cameras requires that the user rotate 

is shown, in which the image's height is greater than its p^^i^^j^ ^^^^^^ ^ ^^^^^ j^^y presented in a more natural 

^ and upright viewing angle on the image display 100. It 

In a digital camera, an image sensor is comprised of should be noted that this problem also exists for digitized 

light-sensitive devices, such as charge-coupled devices images from conventional film cameras. Traditional rotation 

(CCD), that convert an optical image into a set of electrical 40 methods also have the disadvantage of requiring two blocks 

signals. Referring now to FIG. IC, a prior art image sensor of memory lo rotate a stored image, 

is shown having a 480 row by 640 column matrix of light ^^at is needed is an apparatus and method that efiBciently 

collecting pixels. The image sensor is orientated withm a a^d automaticaUy rotates a stored photographic image to 

body of the digital camera so that a first row of pixels (i.e. correspond lo the orientation in which the photographic 

row rj) corresponds to the bottom of an upright and level 45 image was captured, 
digital camera. This image sensor orientation is required 

since as an optical image passes through a conventional SUMMARY OF THE INVENTION 

camera lens it is inverted. The image sensor in a color digital The present invention is an apparatus and method for 

camera typically consists of an array of green (G), red (R) rotating the display orientation of a captured image. The 

and blue (B) pixels. Alternative embodiments include sen- 50 apparatus of the present invention preferably comprises an 

sors detecting cyan, magenta, yellow and green as is typi- image sensor, an orientation sensor, a memory and a pro- 

cally used in video cameras. Other image sensor configu- cessing unit. The image sensor is used for capturing image 

rations are also used. The pixels that comprise the image data. The orientation sensor is coupled to the image sensor, 

sensor are arranged into various patterns or formats. A and is used for generating a portrait signal if the image 

common image sensor format is called a Bayer pattern. The 55 sensor is positioned in a portrait orientation relative to the 

Bayer pattern format is defined as a pixel pattern comprised object. The memory, has an auto-rotate unit comprising 

of 50% green-light responsive pixels, 25% red-light respon- program instructions for transforming the captured image 

sive pixels and 25% blue-light responsive pixels arranged in data into rotated image data in response to the portrait signal, 

alternating rows of "GRGRGR" and "BGBGBG," as shown The processing unit, is used for executing program instruc- 

in FIG. IC. Throughout this specification, "G" means 50 tions stored in the memory, and is coupled to the image 

"green," "R" means "red," and "B" means "blue." sensor, the orientation sensor and the memory. 

Once an image is captured by the digital camera, a set of The method of the present invention preferably comprises 

pixel signals corresponding to the image received by the the steps of: generating image data representative of an 

pixels is processed by an image processing algorithm. Image object with an image sensor; identifying an orientation of the 

processing routines are conventionally designed to process 65 image sensor relative to the object during the generating 

pixel signals line-by-line, conforming to a specific and step; and selectively transferring the image data to an image 

unchanging pixel pattern format. Thus, image sensors manu- processing unit in response lo the identifying step. 
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ESCRIPTION OF THE DRAWINGS 



FIG. JA., i§ a prior art graphical representation of an 
object in aifandscape orientation; 

FIG. IB, iyk prior art graphical representation of the 
object in a Bortrail orientation; 

FIG. Ijc, i^a prior art 480 row by 640 column image 
sensor; y/^ 

FIG. U5, a prior art graphical representation of the 
object as ca|Uured in the portrait orientation and output upon 
an image ^isplay; 

FIG. 2^ is a block diagram of a preferred embodiment of 
an apparatus for rotating the display orientation of a cap- 
tured image/^ 

FIG. 3/Cisa/6iock diagram of a preferred embodiment of 
an imagine^bsystem of the present invention; 

FIG. 3j0y is a first graphical representation of a preferred 
embodiment of an image sensor disposed within the present 
invention; / 

FIG. 30!, is a second graphical representation of a pre- 
ferred einbodiment of the image sensor within the present 
invention; 

FIG. 4AO/ a block diagram of a preferred embodiment of 
a processing and storage subsystem of the present invention; 

FIG. 4^ is a block diagram of a preferred embodiment of 
a volatile memory within the processing and storage sub- 
system; / 

FIG. 4jC,i^a block diagram of a preferred embodiment of 
a frame buner data structure within a frame buffer; 

FIG. is yblock diagram of the preferred embodiment 
of a non-volaole memory within the processing and storage 
subsystem;/ 

FIG. 5A, is^ a. graphical representation of a preferred 
embodimenyof portrait_left image data within the frame 
buffer data/tnyture; 

FIG. 5B, U5 a graphical representation of a preferred 
embodimenyof portrait^^right image data within the frame 
buffer data /tructure; 

FIG. 5^, is a graphical representation of a preferred 
embodiment of lapdscape imag^ data within the frame buffer 
data strucmre; and 7 

FIGS. /a, 6B, 6Q:and 6Q( are a flowchart of a preferred 
method K>r rotating the display orientation of a captured 
image. 

rp^" DETAILED DESCRIPTION OF THE 

PREFERRED EMBODIMENTS 

The present invention is an apparatus and method for 
rotating the display orientation of a captured image. 

Referring now to FIG. 2, a block diagram is shown 
illustrating a camera 200 for rotating a display orientation of 
a captured image. In the preferred embodiment, the camera 
200 is a digital camera, such as a QuickTake 150, by Apple 
Computer of Cupertino, Calif. The camera 200 preferably 
comprises an imaging subsystem 204, a bus 206 and a 
processing and storage subsystem 208. Reflected light from 



Referring now to FIG. 3 A, a block diagram is shown 
illustrating the imaging subsystem 204 of the present inven- 
tion. The imaging subsystem 204 preferably comprises a 
lens 300 having an iris, an optical Alter 302, an image sensor 
304, a timing generator 306, an analog signal processor 
(ASP) 308, an analog-to-digital (A/D) converter 310, a 
digital signal processor (DSP) 320, a right/left orientation 
sensor (RLOS) 324, an up/down orientation sensor (UDOS) 
326 and one or more motors 312. U.S. patent application Sen 
No. 08/355,031, entitled A System and Method For Gener- 
ating a Contrast Overlay as a Focus Assist For An Imaging 
Device, filed on Dec. 13, 1994 is incorporated herein by 
reference and provides a detailed discussion of the preferred 
elements in the imaging subsystem 204, with the exception 
of the RLOS 324 and the UDOS 326. U.S. patent application 
Ser. No. 08/384,012, entitled Apparatus and Method for 
Camera image and Orientation Capture, filed on Feb. 6, 
1995 is incorporated herein by reference and provides a 
detailed discussion of various embodiments of the RLOS 
324 and the UDOS 326, which are further discussed below. 
Briefly, the imaging subsystem 204 captures an image of the 
object 202 via light impacting the image sensor 304 along an 
optical path 314. The image sensor 304 in turn generates a 
set of pixel signals representative of the captured image data. 
After being processed by the ASP 308, the A/D converter 
310 and the DSP 320; the captured image data passes over 
the bus 206 to the processing and storage subsystem 208. 
The functions performed by the DSP 320 may alternatively 
be performed in the processing and storage subsystem 208. 

Referring now to FIG. 3B, a graphical representation is 
shown illustrating the image sensor 304 disposed within the 
camera 200. FIG. 3B shows the camera 200 maintained in a 
landscape orientation while directly facing the object 202. 
The image sensor 304 has a front 338 and a top 336, which 
are juxtaposed with a front 334 and a bottom 332 of the 
camera 200 respectively. The front side 338 of the . image 
sensor 304 comprises light-sensitive pixels, which capture 
an image received along the optical path 314 and through the 
lens 300 mounted upon the camera 200. It should be noted 
40 that the image recorded by the image sensor 304 is inverted 
relative to the object by the lens 300. The front 338 of the 
image sensor 304 preferably forms a planar surface as 
shown in perspective in FIG. 3B. When the image sensor 
304 is oriented as shown in FIG. 3B, a landscape image of 
the object 202 is captured. When the camera 200 is rotated 
either 90* clockwise or 90** counterclockwise about the 
X-axis, a portrait image of the object 202 is captured. 

Referring now to FIG. 3C, a graphical representation is 
shown iUustrating the image sensor 304, which has the top 
50 336, a right side 342, a left side 340 and a bottom 344. The 
image sensor 304 is comprised of a plurality of pixels 
arranged in an i-row by j-column array. In the preferred 
embodiment, the pixels are arranged in a Bayer pattern 
format. Alternatively the present invention may be applied 
to image sensors of various sizes and image processing 
routines designed to process pixel signals arranged in pat- 
terns or formats different from the Bayer pattern. 

As stated above, the RLOS 324 (i.e. the right/left orien- 
tation sensor) and the UDOS 326 (i.e. the up/down orien- 
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an object 202 is conventionally captured by the imaging 60 tation sensor) are disposed within the imaging subsystem 



subsystem 204 via a button press or some other action. The 
captured image data is transferred over the bus 206 to the 

processing and storage subsystem 208, which stores the data 
in internal memory. The bus 206 also passes various status 
and control signals between the imaging subsystem 204 and 
the processing and storage subsystem 208, as will be further 
discussed below. 
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204. Both the RLOS 324 and the UDOS 326 are physically 
coupled to the image sensor 304 and electrically coupled to 
the DSP 320. The RLOS 324 monitors whether the camera 
200 is in a landscape orientation or a portrait orientation. If 
the camera 200 is in a portrait orientation, the RLOS 324 
identifies whether the camera 200 has been rotated clock- 
wise or counterclockwise, as viewed from the rear of the 
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camera 200. Id response to such camera 200 rolation, the The buffer 400 is preferably a first-in, first-out buffer for 
RLOS 324 generates and transmits either a landscape signal, temporarily holding the captured image data and the imag- 
a portraitjeft signal or a portrail_right signal to the DSP ing subsystem 204 control signals passed between the DSP 
320, More specifically, if the camera 200 is held in the 320 and the processing and storage subsystem 208. The 
landscape orientation shown in FIGS. lA and 3B, the RLOS 5 buffer 400 has data lines that are coupled to both the bus 206 
324 generates the landscape signal, since the top 336 of the and the internal bus 402. The processing unit 404 executes 
image sensor 304 corresponds to the "TOP" of the object programming instructions stored in the non-volatile memory 
202. However, if the camera 200 is rotated 90° clockwise 406 and the volatile memory 410 to perform various opera- 
about the x-axis, as viewed from the rear of the camera 200 tions. The non-volatile memory 406 stores a set of computer 
shown in FIG. 3B, the RLOS 324 generates the portrait_ jq readable program instructions for controlling how the pro- 
right signal, since now the right side 342 of the image sensor cessing unit 404 accesses, transforms and outputs the image 
304 corresponds to the "TOP" of the object 202. And, if the data, as described in detail below with reference to FIG, 4D. 
camera 200 is rotated 90® counter-clockwise about the While the non-volatile memory 406 is preferably a Read 
X-axis, as viewed from the rear of the camera 200, the RLOS Only Memory (ROM), alternatively the non-volatile 
324 generates the portrait_left signal, since now the left side memory 406 could be replaced with a functionally equiva- 
340 of the image sensor 304 corresponds to the "TOP' of the lent computer useable medium such as: a compact disk and 
object 202. drive; a floppy disk and drive; or a memory card. Additional 

The UDOS 326 monitors whether the camera 200 is in details on the function and operation of both the non-volatile 

either a level orientation or an off-level orientation. In memory 406 and the volatile memory 410 are provided 

response to a movement of the camera 200, the UDOS 324 20 ^^^^^ ^^^^ reference to FIGS. 4B and 4D. 

generates and transmits either a level signal or an off-level The input device 408 is preferably a series of control 

signal to the DSP 320. Specifically, if the camera 200 is held buttons, responsive to user inputs, which generates signals 

in the landscape orientation shown in FIG. 3B, the UDOS that are translated by the processing unit 404 into control 

324 generates the level signal, since the front 334 of the signals to adjust the focus and other operating parameters of 

camera 200 directly faces the object 202. If the camera 200 25 the imaging subsystem 204. As mentioned above, while the 

is rotated more than 45** clocbvise or counter-clockwise preferred embodiment of the camera 200 incorporates the 

about the y-axis, shown in FIG. 3B, the UDOS 324 prefer- RLOS 324 and UDOS 326, an alternative embodiment of the 

ably generates the off-level signal, since the front 334 of the present invention prompts the user to manually input, via the 

camera 200 now points either above or below the x-y plane. input device 408, the imaging subsystem's 204 orientation at 

Alternative embodiments of the present invention can 30 the time the captured image data was generated. Other 

change the specific measure of rotation necessary to gener- alternative embodiments incorporating the automatic RLOS 

ate the off- level signal from 45® to any number from 0° to 324 and UDOS 326 sensors could provide for an override 

180**. function that instructs the processing and storage subsystem 

Both the RLOS 324 and UDOS 326 sensors are preferably 208 to ignore the orientation signals generated by the RLOS 

identical in their structure and function. Their placement 35 ^24 and UDOS 326 sensors, enabling the user to retain full 

relative to the image sensor 304 determines whether the manual control over the final orientation of the captured 

signal generated by the orientation sensor is one of the image data on the image display 100. The I/O Interface 414 

RLOS 324 signals or' the UDOS 326 signals. The DSP 320 is coupled to the internal bus 402 and has an external port 

receives both the RLOS 324 and the UDOS 326 signals and connector for coupling the processing and storage subsystem 

transmits them to the processing and storage subsystem 208 40 208 with a host computer (not shown) for downloading 

over the bus 206. While the preferred embodiment of the image data stored in the volatile memory 410 and thus 

present invention incorporates the orientation sensors 324, freeing up room in the volatile memory 410 for additional 

326, those skilled in the art will recognize that the orienta- sets of processed image data. In the second and alternative 

tion sensors 324, 326 are not required to practice the present embodiment discussed above where the processing and 

invention, if the processing and storage subsystem 208 45 storage subsystem 208 is a desktop type computer, the input 

prompts the photographer for the camera's 200 orientation at device 408 will also include a keyboard and mouse type 

the time an image is captured. Alternatively, an embodiment controller. 

using only the RLOS 324 enables practicing of the present Referring now to FIG. 4B, a block diagram is shown 

invention, but this alternative embodiment results in loss of illustrating a volatile memory 410 within the processing and 

the off-level signal. However, such an embodiment using 50 storage subsystem 208. The volatile memory 410 contains a 

only the RLOS 324 is still useful for most applications frame buffer 416, working memory 418 and storage memory 

requiring only landscape and portrait orientation using an 420 coupled to the internal bus 402. The volatile memory 

approximately level camera 200. A further alternative 410 is preferably a Random Access Memory (RAM), 

embodiment includes an RLOS 324 which only generates The frame buffer 416 is preferably comprised of a dcdi- 

the landscape, portrait_left, or portrait__right signals if the 55 cated space of contiguous memory comprising an i-row by 

camera 200 is within a predetermined level range. j-column area for storing image data. There is preferably a 

Referring now to FIG. 4A, a block diagram is shown constant one-to-one correspondence between the pixels 

illustrating the processing and storage subsystem 208 of the comprising the image sensor 304 array and the storage 

present invention. The processing and storage subsystem locations within the frame buffer 416 area. Alternatively, the 

208 comprises a buffer 400, a processing unit 404, a non- 60 pixel signals from the image sensor 304 array could be 

volatile memory 406, an input device 408, a volatile stored within the frame buffer 416 in a non-contiguous (i.e. 

memory 410 and an I/O interface 414, coupled together via sectored) memory format, as is commonly known in the art. 

an internal bus 402. In the preferred embodiment, the In the preferred embodiment, i and j are set equivalent to the 

processing and storage subsystem 208 is conventionally size of the image sensor 304. Since the preferred image 

embedded as part of the camera 200. Alternatively, the 65 sensor 304 is a 481 by 641 array, i is set to 481 and j is set 

processing and storage subsystem 208 may be a desktop to 641. In alternative embodiments where the image sensor 

type computer. 304 is only a 480 by 640 array, the frame buffer 416 will still 
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preferably be a 481 by 641 array since, as is discussed Hardware Embodiment 

below, an extra row and column is generated by the pro- , In the hardware embodiment, the image sensor 304 is 

cessing and storage subsystem 208. comprised of at least one more extra row and column than 

The working memory 418 preferably provides a tempo- the image processing unit 424 is designed to process. Thus, 

rary space for storing data received from the input device 5 if the image processing unit 424 is designed to process a 480 

408 and for data generated by the processing unit 404. The by 640 pixel signal array, then the image sensor 304 must be 

working memory 418 also contains additional memory for at least a 481 by 641 pixel array (i.e. "i" is 481 and "j" is 641 

storing the RLOS 324 and UDOS 326 signals at the time the i„ piQ. 3C). The discussion that follows on the auto-rotate 

image data was captured by the image sensor 304; and ^^it 422 makes clear when and why the additional row and 

additional information conventionally associated with the column is required 

capture process, such as exposure and focus settings time Software Embodiment 

and date, statistical data, and me presence ot any detective ^^^^^^^ embodiment is required when the image 

pixels, rows or columns within the image sensor 304. - , , jjv- 1 j 1 i_ j 

^ ^ ^-^^ r . , . , r sensor 304 lacks any additional rows and columns beyond 

Hie storage memory 420 preferably provides space for ^^^^ ^^^^ • processing unit 424 is designed to 

storing multiple sets of image data, unti the images are ^3 ^^^^ g ^^^^^ . processing, the image 

off-loaded from the camera 200 via the I/O interface 414. ^ ^^j^ 428 must generate an additional row and 

The storage memory 420 sets a ntiaxmium miage data ^^j^^^ ^ ^^^^ (P^^ non-Bayer pattern image data, a 

holdmgcapacityof lhecamera200 AlteraaUvely,addition^ straightforward rotation is performed as is described below 

memory cards could also be coupled to the internal bus 402, ^„ additional row and column is not required.) Thus, for 

providing even more image data storage capabihty Such an ^^^^ • ^^^^ comprised of a 480 by 

additional memory card could be a non-volatile flash p^^^j ^^^^ -^^^^ processing unit 424 is 

memory card. designed to process a 480 by 640 array, then the image 

Referring now to FIG. 4C, a block diagram is shown capture unit 428 must generate an additional row and 

illustrating a memory map of a volatile memory data struc- column of pixels. In the preferred software embodiment, the 

ture 417 within the volatile memory 410. The volatile ^ additional row generated corresponds to row r, (i.e. 

memory data structure 417 comprises an unage data field gRGRGR) in FIG. 3C and the additional column generated 

419 for storing the captured image data, an RLOS signal corresponds to column c^- (i.e. GBGBGB) in FIG. 3C. The 

data field 421 for storing the RLOS 324 signal, an UDOS additional row and column may be generated in any number 

signal data field 423 for stonng the UDOS 326 signal and a ^^y^,^ including: setting the additional pixels to black; 

miscellaneous data field 425 for stonng the exposure and 30 creating a dupHcate of a nearby line of pixels with the same 

focus settmgs, time and date, statistical data, and the pres- pi^d pattern (i.e. two pixel lines over); generating a dupli- 

p ence of any defective image sensor 304 pixels, rows or ^^^^ ^^^,^5^ color (i.e. G's would be copied from one 

fU columns. The interactions of each of all these data fields 419, pi^el line over while R*s or B's would be copied from two 

421, 423, 425 with the camera 200 is described in detail pi^d lines over); setting the pixel values based on a filtering 

"L. below. 35 algorithm which looks at a number of nearby pixels; or 

Vy Referring now to FIG. 4D, a block diagram is shown setting the additional row and column of pixels randomly, 

s illustrating the non-volatile memory 406 within the process- jhe auto-rotate unit 422 transforms the captured image 

£3 storage subsystem 208. The non-volatile memory data into the rotated image data for further processing by the 

406 comprises an auto-rotate unit 422, an image processing image processing unit 424. The image processing unit 424 

iZ ^"^^ image capture unit 428 and an operating system 40 receives the rotated image data from the auto-rotate unit 422 

1=4 426. and transforms it into processed image data, ready for saving 

^£ The image capture unit 428 controls the imaging sub- in the storage memory 420. The operating system unit 426 

7=1 system's 204 focus and aperture settings; captures the image is coupled to the internal bus 402 and preferably comprises 

If; data; routes the image data into the frame bufifer 416; routes a conventional set of program instructions for controlling the 

the exposure and focus settings, time and date, statistical 45 configuration and usage of the computer's 208 hardware 

data, and the presence of any defective image sensor 304 resources. The operations performed by the auto-rotate unit 

pixels, rows or columns into the miscellaneous data field 422 and the image processing unit 424 are described in more 

425; generates additional rows and columns of image data detail below. 

within the frame buffer 416 if required; monitors the RLOS Referring now to FIGS. 5A, 5B and 5C, a graphical 

324 and UDOS 326; and sets the RLOS signal 421 data field 50 representation is shown in FIG. 5A illustrating portraitjeft 

and the UDOS signal 423 data field based on the RLOS 324 image data within the frame buffer 416; a graphical repre- 

and UDOS 326 outputs. In the preferred embodiment the sentation is shown in FIG. SB illustrating portrait_right 

image capture unit 428 corrects any pixel defects within the image data within the frame bufifer 416; and a graphical 

image sensor 304 using the information stored in the mis- representation is shown in FIG. 5C illustrating landscape 

cellaneous data field 425 before passing the image data to 55 image data within the frame buffer 416. FIGS. 5A, 5B and 

the auto-rotate unit 422. In an alternative embodiment, 5C all show the same image data 419 within the frame buffer 

image processing unit 424 corrects any pixel defects within data structure 417 filled with different sets of captured image 

the image sensor 304 using the information stored in the data and graphically positioned in the orientation in which 

miscellaneous data field 425 after receiving the image data the camera 200 was held at the time the captured image data 

from the auto-rotate unit 422. In this alternative 60 was generated. As will be discussed in detail below, the 

embodiment, the defective image sensor 304 pixel informa- shaded areas represent a subset of pixel signals within the 

tion must also be rotated by the auto-rotate unit 422 in the image data field 419 that the auto-rotate unit 422 selects to 

same manner as for the image data, described below. be sent to the image processing unit 424 as the rotated image 

The image capture unit 428 preferably is designed in two data, 

embodiments, a hardware embodiment and a software 65 The auto-rotate unit 422 preferably comprises program 

embodiment; however, the hardware embodiment is pre- instructions stored in the non-volatile memory 406 that are 

ferred. executed by the processing unit 404. The auto-rotate unit 
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422 controls the flow of data between the frame buffer 416 data field 419 location (r^.^, Cj) to the image processing unit 

and the image processing unit 424. First, the aulo-rotation 424. Decrementing the row number by one, the portraitjeft 

unit 422 accesses the RLOS signal 421 data field and UDOS spooling routine next transfers the pixel signal at location 

signal 423 data field from the frame buffer data structure (r,_2, C2) to the image processing unit 424, and so on until 

417; and, based on this orientation data, selects either a 5 pixel (r^, C2) has been transferred. The portrait left spooling 

portrait_left spooling routine, a portrait_right spooling routine then increments the column by one to C3 and resets 
routine or a landscape spoohng routine. The spooling rou- the row to r-.^ in preparation for spooling the next line of 
tines select -out and feed the captured image data stored in pixel signals (having a length of i-1) to the image processing 
the image data field 419 to the image processing unit 424. unit 424, in the same manner as the first line (having a length 
"Spooling" is here defined as a routine for transferring data 10 of i-1) was spooled. The portrait_left spooling routine 
from one unit or device to a second unit or device. It is continues to increment the column number and spool the 
through these spooling routines that the auto-rotate unit 422 next line of pixel signals (having a length of i-1) to the 
transforms the captured image data into the rotated image image processing unit 424 until column Cy has been spooled, 
data. While the effect of the auto-rotate unit 422 is only The pixel signals within either row r^ or column c^ are not 
described below with reference to the captured image data, 15 sent to the image processing unit 424 since the image 
some of the additional information stored in the working processingunit424ispreferably designed to process only an 
memory 418 (such as the location of any defective pixels, i-1 by j-1 array of pixel signals. As a result of this spoofing 
rows or columns within the image sensor 304) will also be sequence, the entire set of pixel signals representing a 
rotated, as described above. This additional information is portrait_left image has been transferred to the image pro- 
then used by the image processing unit 424. Alternatively, a 20 cessing unit 424. Furthermore, due to this specific spooling 
defective pixel correction algorithm can be applied to the sequence, the portrait left image will be displayed on the 
image data prior to rotation of the image data. Each of the image display 100 so that the "TOF* of the portraitjeft 
different spooling routines are described in detail below. image corresponds to the top 102 of the image display 100.. 

First, the auto-rotate unit 422 determines the image sen- If the portrait_right spooling routine is selected, the 

sor*s 304 orientation at the lime the image was captured, by 25 auto-rotate unit 422 interprets the captured image data 

accessing the RLOS signal 421 and UDOS signal 423 data within the image data field 419 as shown in FIG. 5B. FIG. 

fields. Next, if the RLOS signal 421 data field contains a 5B shows an i-row by j-column set of captured image data 

portrait_left signal and the UDOS signal 423 data field in a portrait_right orientation. As shown, the "TOP" of the 

contains a level signal, the auto-rotate unit 422 selects the portrait_right orientation corresponds to column Cy.^. A 

portrait_left spooling routine. However, if the RLOS signal 30 subset of the image data field 419 is selected by the 

421 data field contains a portrait_right signal and the UDOS portrait__right spooling routine, as shown by the shaded gray 

signal 423 data field contains a level signal, the auto-rotate area in FIG. 5B, to be sent to the image processing unit 424. 

unit 422 selects the portrait_right spooling routine. Lastly, The shaded area of FIG, 5B is defined from rows r2 to r^-, and 

if the RLOS signal 421 data field contains a landscape signal from columns c^ to Cy_i. The total array size of the shaded 

and the UDOS signal 423 data field contains a level signal, 35 area is therefore comprised of i-1 number of rows and j-1 

or whenever the UDOS signal 423 data field contains an number of columns. Next, the portrait_right spooling rou- 

off-level signal, regardless of what the RLOS signal 421 data tine defines the shaded area of Cy_i as the top line of the 

field contains, the auto-rotate unit 422 selects the landscape rotated image data and configures the image processing unit 

spooling routine. Each of the three spooling routines within 424 for a line length of i-1 pixels. The portrait_right 

the auto-rotate unit 422 are now discussed below. 40 spooling routine first transfers the pixel signal in the image 

If the portrait__left spooling routine is selected, the auto- data field 419 location (r2, Cy.j) to the image processing unit 
rotate unit 422 interprets the captured image data within the 424. Incrementing the row number by one, the portrait^ 
image data field 419 as shown in FIG. 5A. FIG. 5A shows right spooling routine next transfers the pixel signal at 
an i-row by j-column set of captured image data in a location (rg, Cy.^) to the image processing unit 424, and so 
portrait_left orientation. As shown, the "TOP" of the 45 on until pixel (r,-, Cy_i) has been transferred. The portrait_ 
portraitjeft orientation corresponds to column C2. Note that right spooling routine then decrements the column by one to 
the "row" and "column" designations defined for the image Cy_2 and resets the row to r2 in preparation for spooling the 
sensor 304 in FIG. 3C are retained for clarity in FIGS. 5A, next line of pixel signals (having a length of i-1) to the 
5B and 5C even though the camera has been rotated into the image processing unit 424, in the same manner as the first 
portraitjeft orientation. Conventional image processing 50 line (having a length of i-1) was spooled. The portrait_right 
units are designed to receive image data in a constant pixel spooling routine continues to decrement the column number 
pattern format. Using the Bayer pattern format (assumed for and spool the next line of pixel signals (having a length of 
the purposes of this discussion) the image processing unit i-1) to the image processing unit 424 until column Cj has 
424 must receive the image data from the image data field been spooled. Thus, the entire set of pixel signals represent- 
419 starting with a "GRGRGR" line and followed by a 55 ing a portrait_right image has been transferred to the image 
"BGBGBG" line, in an alternating hne-by-line pattern. processing unit 424. As a result of this specific spooling 
Therefore, a subset of the image data field 419 is selected by sequence, the portrait_right image will be displayed on the 
the portrait_left spooling routine, as shown by the shaded image display 100 so that the "TOP" of the portrait_right 
gray area in FIG. 5A, to be sent to the image processing unit image corresponds to the top 102 of the image display 100. 
424. The shaded area of FIG. 5Ais defined from rows r^ to 60 If the landscape spooling routine is selected, the auto- 
r-_i, and from columns C2 to Cy. The total array size of the rotate unit 422 interprets the captured image data within the 
shaded area is therefore comprised of i-1 number of rows image data field 419 as shown in FIG. 5C. FIG. 5C shows 
and j-1 number of columns. Next, the portrait_left spooling an i-row by j-column set of captured image data in a 
routine defines the shaded area of C2 as the top line of the landscape orientation. As shown, the "TOP" of the land- 
rotated image data and configures the image processing unit 65 scape orientation corresponds to r^. A subset of the image 
424 to accept a line length of i-1 pixels. The portraitjeft data field 419 is selected by the landscape spooling routine, 
spooling routine first transfers the pixel signal in the image as shown by the shaded gray area in FIG. 5C, to be sent to 
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the image processing unit 424. The shaded area of FIG. 5C decrement the column to c^.j, spool column c^-.j and so on; 

is defined from rows r^ to r,_j, and from columns c^ to Cy_^. and the landscape spooling routine would start with pixel 

The total array size of the shaded area is therefore comprised signal (rj, cj, spool row r^, increment the row to r^, spool 

of i-1 number of rows and j-1 number of columns. Next, the row rj and so on. In all three cases above, the extra or 

landscape spooling routine defines the shaded area of as 5 provided row r,- and extra or provided column Cy would not 

the top line of the rotated image data and configures the be transferred. 

image processing unit 424 for a line length of j-1 pixels. Referring now to FIGS. 6A, 6B, 6C and 6D, a flowchart 

Alternatively, the shaded gray area representing the rotated of a preferred method for rotating the display orientation of 

image data could be positioned over different sets of pixel a captured image is shown. The method begins in FIG. 6A 

signals within the image data field 419, again subject to the lO with step 600 where the image capture unit 428 captures a 

constraint that the top line begin with the "GRGRGR" set of image data and a set of image sensor 304 orientation 

pattern. The landscape spooling routine first transfers the data approximately simultaneously. The RLOS 421 and 

pixel signal in the image data field 419 location (r^, c^) to the UDOS 423 orientation signals are recorded when the image 

image processing unit 424. Incrementing the column num- data capture occurs in order to insure that the orientation 

ber by one, the landscape spooling routine next transfers the 15 sensor information accurately reflects the actual position of 

pixel signal at location (r^, c^) to the image processing unit the image sensor 304. In step 602, the image capture unit 

424, and so on until pixel (r^, Cy.^) has been transferred. The 428 transfers the captured image data and the image sensor 

landscape spooling routine then increments the row by one 304 orientation data into the volatile memory 410. In Step 

to T2 and resets the column to c^ in preparation for spooling 603, the image capture unit 428 generates an additional row 

the next line of pixel signals (having a length of j-1) to the 20 and column of image data if the captured image data does 

image processing unit 424, in the same manner as the first not already include an extra row and column of image data, 

line (having a length of j-1) was spooled. The landscape In Step 604, the auto-rotate unit 422 determines the orien- 

spooling routine continues to increment the row number and tation of the captured image data. If in step 606 the captured 

spool the next line of pixel signals (having a length of j-1) image data is in a porlrait_left orientation, the method 

to the image processing unit 424 until row r.-.j has been 25 proceeds to step 612, otherwise the method continues to step 

spooled. Thus, the entire set of pixel signals representing a 608. If in step 608 the captured image data is in a portrait^ 

landscape image has been transferred to the image process- right orientation, the method proceeds to step 628, otherwise 

ing unit 424. As a result of this specific spooling sequence, the method proceeds to step 644. 

the landscape image will be displayed on the image display In step 612 of FIG. 6B, the auto-rotate unit 422 configures 

100 so that the "TOP" of the landscape image corresponds 30 the image processing unit 424 to accept an image data line 

to the top 102 of the image display 100. length corresponding to a portrait_left image. In step 614, 

Also as introduced above, the image processing unit 424 the auto-rotation unit 422 initiahzes a variable named "col- 

preferably comprises program instructions stored in the umn" stored in the working memory 418 to a first pixel 

non-volatile memory 406 that are executed using the pro- column (i.e. to column cj, within the image data field 419, 

cessing unit 404. The image processing unit 424 receives the 35 required by the image processing unit 424. In step 616, the 

rotated image data from the auto-rotate unit 422, performs auto-rotation unit 422 initializes a variable named "row" 

conventional image processing operations (such as white stored in the working memory 418 to a row (i.e. to row r^-_^) 

balancing, reconstruction, color correction, gamma within the image data field 419 containing a first pixel color 

correction, sharpening and compression) on the rotated required by the image processing unit 424. Next, in step 618, 

image data; generates processed image data; and either saves 40 the auto-rotate unit 422 transfers the captured image data in 

the processed image data to the storage memory 420 or the image data field 419 location (r^^, c^oiumrd ^ the image 

outputs it to a host computer (not shown) via the I/O processing unit 424. In step 620, the auto-rotate unit 422 

interface 414. As was discussed above, the image processing decrements "row" to a row containing a next pixel color 

unit 424 is preferably implemented such that it must receive required by the image processing unit 424. In step 622, if 

all the image data spooled from the frame buffer 416 starting 45 "last row"* has been transferred, the method proceeds to step 

with a "GRGRGR" line followed by a "BGBGBG" line in 623, otherwise the method returns to step 618. In the 

an alternating line-by-line .pattern. This is the image pro- preferred embodiment "last row" is "r^" as shown in FIG. 

cessing unit's 424 design limitation which drives the auto- 5 A. When "last row" has been transferred in step 622, an 

rotate unit's 422 particular pixel selection routines described entire line of portrait__left image data has been transferred to 

above. Alternatively, a different image processing unit 424 50 the image processing unit 424. In step 623 the image 

could be implemented which requires that the image data be processing unit 424 performs any required image processing 

spooled from the frame buffer 416 starting with a "GBG- on the transferred image data. Step 624 then increments 

BGB" line followed by a "RGRGRG" line or any combi- "column" to a next pixel column required by the image 

nation of the Bayer pattern. processing unit 424. Finally, if in step 626 "last column" has 

Those skilled in the art will also recognize that alternate 55 not yet been transferred the method returns to step 616, 

embodiments of the present invention that do not use the otherwise the method ends. In the preferred embodiment 

Bayer pattern may still use the teachings of present inven- "last column" is "Cy," as shown in FIG. 5A. 

tion to rotate their image data. In such alternative embodi- In step 628 of FIG. 6C, the aulo-rolate unit 422 configures 

ments the auto-rotate unit 422 performs a straightforward the image processing unit 424 to accept an image data line 

rotation of the image data since such alternative image 60 length corresponding to a portrait_right image. In step 630, 

processing units 424 may accept pixel signals in any orien- the auto-rotation unit 422 initializes a variable named "col- 

tation. Thus for the straightforward rotation where the umn" stored in the working memory 418 to a first pixel 

shaded gray area of FIG. 5C represents the image data, the column (i.e. to column c^.^), within the image data field 419, 

portrait left spooling routine would start with pixel signal required by the image processing unit 424. In step 632, the 

(r^.^^i, c^), spool column Cj, increment the column to C2, spool 65 auto-rotation unit 422 initializes a variable named "row^' 

column C2 and so on; the portrait_right spooling routine stored in the working memory 418 to a row (i.e. to row Tj) 

would start with pixel signal (r^, Cy.,)» spool column Cy^j, within the image data field 419 containing a first pixel color 




